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Proceedings of the Workshop on Benefits Assessment Protocols 


October 21-23, 1997 Raleigh, N.C. 


Executive Summary 


A workshop held in Raleigh, North Carolina, on October 21-23, 1997, was conducted to provide 
guidelines for the development of protocols for pesticide benefits assessments for regulatory purposes. The 
participants included university scientists and extension specialists, together with individuals providing 
perspectives from commodity organizations, government (USDA and EPA), agribusiness, and grower concerns. 
A full list of participants, workshop agenda, and discussion topics is available in the full report available 
through USDA/ARS/NAPIAP or the NSF Center for Integrated Pest Management. 


The following represents a brief overview of the recommendations of the Workshop. Participants were 
in general agreement and all have had the opportunity to review this document. 


Primary Recommendation 


The workshop participants recommend the establishment of an ad hoc group responsible to 
USDA/NAPIAP and EPA. This group should have representatives from all the major stakeholder 
organizations. The charge to this ad hoc group would be to prepare recommendations for specific protocols, 
designed to estimate pest damage (crop loss), based on the guidelines developed during this workshop. 


Specific protocols should address (1) database and information types and sources, (2) data quality 
assurance measurements, (3) data analysis procedures and output structure, and (4) report and analysis review 
and response. These protocols should provide a standardization of requirements for benefits assessments based 
on scientific evidence and commonly accepted methods of data analysis and interpretation. The rationale for 
the development of these protocols is presented in brief below and in more detail in the full proceedings. 


Protocol Development Guidelines 


Information/Databases — Types. The following types of information should be available for benefits 
assessments and be included as part of the protocols. 


Inventory of relevant scientists (by state, commodity, discipline, other areas of expertise). This inventory 
should be publicly accessible. 


Pest Database. This should include crop loss data/models, resistance information, and 
geographic/seasonal incidence. 


Pesticide Database. This should include a list of approved or preferred information sources including 
specific journals and databases(by crop), pesticide alternatives, efficacy, and economic information (market 
share, usage, price). 
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Benefits Protocols Workshop 


Crop Statistics. This should include, at the least, acres planted, distribution, price, and quality measurements 
(e.g., oil content, fresh market versus canning quality, where appropriate). 


Data from each of the latter three types (pest, pesticide, crop) are required to estimate crop loss and 
potential net economic benefits from specific pesticide use. 


Information/Databases — Sources. Regardless of the source, it should be a requirement that the information be 
identified as to source documentation, authors, methodologies used, and quality (see next section for more on 
quality). Priority should be given to information from published (peer-reviewed) databases/documents, followed 
by surveys/questionnaires, unpublished hard (experimental) data, and, lastly, scientific estimates. Protocols 
based on this approach can be scientifically justified, while still allowing flexibility where experimental data 
are lacking, contain divergent results, or are of disputed value. 


Quality Assurance. Specific quality standards criteria must be developed for varying types of data. The criteria 
may be different for experimental data as opposed to surveys, questionnaires or scientific estimates. As an 
example, for experimental data, the workshop participants recommended the use of a quality index (Borth 
Index, see page 7 of full proceedings) which considers in situ replication, together with replication across 
regions and growing seasons. Data and databases which have higher levels of replication can be weighted more 
heavily in the decision process than similar information with less replication through time or geographically. 
Developing protocols for information quality, as outlined above, are a necessity for accurate and reproducible 
benefits assessments. 


Analysis/Output Structure. Most importantly, criteria for protocols in this area must address documentation of 
any analysis or modeling. Any output which alters the data or databases used must be documented and the 
results reproducible (i.e., results can be recreated). Protocols in this focus area must also define output shell(s) 
or template(s) for defined user groups (e.g., EPA decision-makers or commodity group analysts). For regulatory 
purposes, the analysis/model output should be in a form with direct utility to the regulatory decision process, 
with no or minimum alteration required. This will reduce response time and potential misinterpretations of 
analysis/model results. 


Peer Review. These protocols must be established to determine who reviews assessments and how they are 
reviewed. Such protocols will also protect all affected parties. All stakeholders should be represented in the 
review process (registrants, state agricultural extension services, commodity groups, regulatory agencies, and 
environmental/safety concerns). These protocols must consider appropriate experts from the categories above, 

actual time constraints, conflict resolution, and feedback to/from the reviewers. Because all data and analyses 
are in part subjective, peer review should be a prerequisite to any benefits assessment and review protocols 


developed. 
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Benefits Protocols Workshop 


Attendees* 
Invited Participants 


Michael Aerts, University of Florida (Agronomy) 

John Ayers, Pennsylvania State University (Plant Pathology) 
Ron Davis, ARS (Plant Pathology) 

Leonard Gianessi, National Center for Food and Agricultural Policy (Economics and Policy) 
Paul Gillebeau, University of Georgia (Entomology) 

Linda Herbst, University of California, Davis (Agronomy) 
Lynnae Jess, Michigan State University (Agronomy) 
Norman Nesheim, University of Florida (Plant Pathology) 
Robert Nichols, Cotton Incorporated (Research Policy) 
David Pike, University of Illinois (Agronomy) 

Larry Olsen, Michigan State University (Entomology) 
Donald Rutz, Cornell University (Entomology) 

Ken Sorensen, North Carolina State University (Entomology) 
Steve Toth, North Carolina State University (Entomology) 
Maury Weise, University of Idaho (Plant Pathology) 


Moderator/Facilitator 
Ron Stinner, NSF Center for IPM, North Carolina State University 


Speakers 


Leonard Gianessi, National Center for Food and Agricultural Policy (Economics and Policy) 
Charles Main, North Carolina State University (Plant Pathology) 

Robert Nichols, Cotton Incorporated (Research Policy) 

David Pike, University of Illinois (Agronomy) 

Ronald Stinner, North Carolina State University (Entomology) 

Gail Wilkerson, North Carolina State University (Agronomy) 


Stakeholder Representatives** 


EPA — David Brassard 

USDA/ARS/NAPIAP — Nancy Ragsdale 

USDA/CSREES/NAPIAP — Dennis Kopp 

Industry — Paul Borth, Dow AgroSciences 

Commodities — Robert Nichols, Cotton Incorporated 

Non-Profit Research — Leonard Gianessi, National Center for Food and Agricultural Policy 


* Individual attendees may be listed in more than one group. 
** Stakeholder representatives served as information sources and provided sector perspectives. 
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Benefits Protocols Workshop 4 


Agenda for 
Benefits Assessment Protocols Workshop 


October 21-23,1997 
Raleigh, North Carolina 


Tuesday, October 21 


1:00 - 2:30 pm Welcome - Ron Stinner, CIPM 
Introduction/Background Review - Nancy Ragsdale, NAPIAP 
FQPA Impact/Risk vs. Benefits - Dave Brassard, EPA 
Charge to Work Groups - Nancy Ragsdale - NAPIAP 


2:30 - 3:00 pn BREAK 


3:00 - 5:00 pmThree work groups will meet: each group focusing on a different pest 
category (insect, disease, weed). Their charge will be to establish 
parameters for yield variability, the rationale for geographic 
specificity. The groups will discuss what protocols can be quickly 
and uniformly established. How can benefits (such as yield 
primarily) impacts be compartmentalized and identified to make 
the process transparent, that is, with justification and 
accountability. 


Wednesday, October 22 


8:00 - 9:45 am Work group reports and group discussion - what protocols are reasonable 
to establish for regulatory purposes? Have any major 
considerations been ignored? 


9:45 - 10:00 am BREAK 


10:00 - Noon What’s available? Explanations and “hands-on” demonstrations of 
several databases and approaches - Charles Main, Gail Wilkerson, 
Leonard Gianessi, Robert Nichols, David Pike, Ron Stinner 


12:00 - 1:00 pm LUNCH 


1:00 - 4:00 pm Work groups separate again to do a “benefits analysis” walk-through. The 
insect group will have a specific problem on crucifers, the 
disease/nematode group will have grapes, and the weed group will 
have soybean. 
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Benefits Protocols Workshop 


The selection of crops is based on wanting to provide a breadth of 
problems. Crucifers represent a “limited-use” crop with major 
resistance problems. Grapes are a perennial crop with a wide 
distribution and both “table” juice and wine markets. Soybean, 
with new genetically engineered varieties, provide a major field 
crop. 


As the work groups develop their benefit analysis, they will be 
asked to emphasize the needs and protocols which should be of a 
general nature and any which might be pest or crop-type specific. 


4:00 - 5:00 pmEach work group will develop a report on how they achieved their benefits 
analysis. What information was critical, what data useful, what 
information was needed but not available, and what protocols 
could be established to ensure a scientifically justified (transparent) 
assessment. Each group will be asked to prioritize their suggested 
protocols, reflecting how meaningful and achievable the resulting 
data should be for regulating purposes. A written report will be 
due the following morning. 


Thursday, October 23 
8:00 - 9:30 am Each group will have a half-hour report time. 
9:30 - 10:00 am BREAK 


10:00 - Noon The entire group will identify “necessary” protocol areas and 
group-specific protocols. The entire group will be expected to 
develop a prioritized list of general protocols and recom- 
mendations for further consideration by NAPIAP, EPA, and 
stakeholder groups. 


During the Workshop, a recorder will be provided to keep an accurate record of 
discussions. The group reports and abstracted discussions will be compiled into a Proceedings, 
to be made available in electronic and printed form to NAPIAP prior to December 31, 1997. 
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Discussion I Synopsis 


Workshop participants were separated into three discussion groups with general responsibility for 
weeds, insects, and plant pathogens. Participants were placed into groups according to expertise. The three 
groups were: 


Weeds Insects Plant Pathogens 
Giannessi Borth Aerts 

Pike, Chairman Toth Ayers, Chairman 
Herbst, Recorder Gillebeau, Chairman Weise 

Nichols Rutz, Recorder Neisheim, Recorder 
Kopp Sorensen Olsen 

Jess Davis 


Drs. Nancy Ragsdale (USDA/ARS/NAPIAP), David Brassard (EPA), and Ronald Stinner (NSF Center 
for Integrated Pest Management) served as resource personnel and rotated among the three groups. It is 
important to note that ‘benefits’ assessment is only part of the review process, which includes both risk and 
benefits review. 


Historically, the term ‘benefits’ in this process has connoted the ‘impacts,’ including both direct benefits 
as well as potential losses due to uncontrolled pests, while potential health and safety problems were addressed 
under the ‘risk assessment’ part of the review process. We have maintained this approach for the Workshop. 


The three groups were charged with several general tasks. First , the groups were asked to identify the 
critical parameters responsible for yield variability and to explain which of those parameters would be expected 
to have a geographic component. This latter aspect was requested as a justification for regional specificity in 
assessments and/or protocols. The groups were also asked to discuss what protocols could be quickly and 
uniformly agreed without regard to pest ‘types’ (weeds, insects, diseases). 


The groups were also asked to discuss how quantifiable impacts (primarily yield at this point) can be 
compartmentalized and identified to make the benefits assessment process more transparent (that is, contain 
justification and accountability within the process). A synopsis of each group’s report and the resulting 
discussion is presented below. 


Plant Pathogens 


The key variables required include paired-comparisons of treatments, contro] alternatives, disease 
incidence or severity and time-course with geographic and year replication, varieties used (including tolerance 
and/or resistance level), agronomic/horticultural information on varieties, and yield and quality or marketability 


(if appropriate). 


Information and analyses are needed on a regional basis because of differences in climate, pathogen 
species and races of the same species, varietal preferences by the local markets/growers, and divergent cultural 


perspectives. 
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There was a discussion of the need for, and expenses of, standardization of experiments, with examples 
of national programs in tomato and alfalfa diseases. Only the very highest priority diseases could be examined 
and this would have to be an on-going program so that new chemicals and techniques could be evaluated. It was 
pointed out that even with such standardized experimental protocols, the tomato benefits assessment team still 
had to interpret much of the data. 


A discussion of standardization noted that both the Fungicide and Nematicide Trials and the Arthropod 
Management journals require relatively standard formats, but not experimental protocols. If standardization is 
not possible, conversion factors may be plausible. The group agreed that if standardization is not possible (or 
desirable?), then the issue of protocols for quality assurance is critical. 


Insect Pests 


The critical data include yield as the bottom line, in addition to pest numbers and regional incidence. 
The insect group agreed with the plant pathologists on the rationale for regional assessments. The group felt that 
there were two major sources of the needed experimental data. Arthropod Management ( formerly Insecticide 
and Acaricide Trials) is the major U.S. journal and easy to obtain. The second source is the myriad of 
unpublished (or limited publication) reports by State Experiment Stations, State Cooperative Extension Services 
publications and Industry Cooperative Agreement reports. In addition, there are the “file drawer” databases, 
containing many years of hard experimental data which have not been published. Generally, this last category 
may not be available except in extreme circumstances (e.g., EPA threatens a cancellation of a “key” pesticide). 
A database shell available country-wide would enhance the chances of obtaining this data through the state 
PIAP programs. The diversity and variable quality of these data mandate that any protocols developed consider 
some measure of quality. For experimental data, the following schema is proposed (Table 1): 


Table1. The Borth Index. This proposed schema for describing the general quality of experimental data was 
suggested by Dr. Paul Borth of Dow AgroSciences. For simplicity and clarity we will refer to this approach as 
the “Borth Index”’. 


Seasonal Quality/ 
Reliability Index In situ Geographical (years) Value 
Continuum Description Replication Replication Replication Continuum 
HIGH Category I HIGH 

Repeatable X x X 

Category II i x -- 

May Be x ~~ ne 

Repeatable -- X x 

Category III X -- 

Circumstantial -- -- x 


(i.e., demonstrations) 


Category IV -- 34 = 
LOW Not Replicated LOW 
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Our use of the Borth Index references the general concept as outlined and not specific details, which 
would need to be developed as different protocols for different types of data (e.g., experimental data, surveys, 
questionnaires, etc). With the need to both plan for the future as well as deal with emergency requirements, 
workshop participants agreed with the justification for including all available data. This increases the need for a 
quality measure based on generally accepted scientific principles. The Borth Index approach should also apply 
to datasets as well as individual experiments. 


Weeds 


The group reached a number of tenets regarding herbicide benefits assessments. Foremost is the belief 
that because much herbicide ends up in water systems, FQPA will have a major impact. Efficacy trials require a 
long list of variables including those already mentioned, plus a long list of soil measures. To obtain the needed 
yield and quality effects requires many studies over a wide set of conditions. Because of potential yield- 
enhancing effects, such trials need to be taken to final yield. 


The impacts of weeds are particularly difficult to ascertain due to multi-species complexes, and the 
carry-over and yield-enhancing effects of some herbicides. Many of the older datasets (10+ years?) do not have 
standards, but there are two major standardized databases — PDMP and Field Pro. These databases and the 
farmer survey databases used by EPA may be reasonable for major crops, but don’t contain the information 
needed for minor use benefits. 


Yield parameters needed include weighted average of mean yields for 5 years, using blocks of field tests. 
Evaluate those results which are out of the average range with an explanation of the reason for not using a 
particular data point. Use relative yield rather than actual yield. 


One of the difficulties in old efficacy data is that most of the trials do not go to yield. Therefore, yield is 
not known. Can we utilize old data? There were many concerns expressed that the information from old 
studies is not complete. We may need to set up new trials so all the data points necessary in a model would be 
collected. There seems to be a difference in the number of data points necessary to reach an average in minor 
crops and major crops. Minor crops may need significantly fewer than major crops. 


Some of the data needed are: mean yield of a field trial, weeds present, weed density, and any 
environmental conditions which may have had an effect. This is important when plugging data into a model. A 
major problem is that the majority of weed scientists know how effective a chemical is but do not really know 
what the weed pressure is. 


With the continuing fast changes occurring in the control tools available, it was felt that a lot of the older 
data available may be outdated. We need to include not just chemical control but also non-chemical in new 
field trials. New data is where the weight of the evidence should be considered. 


Variables that need to be included in a protocol are: soil textures, pH values, moisture, temperature, 
daily rainfall (+ 2 weeks of planting), tillage method, weed densities, planting rates, planting dates, economic 
thresholds, and yield and quality: (scouting, comparative chemical controls, grading and quality standards). 
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Benefits Protocols Workshop 2 


Discussion II Synopsis 


The same groups set up for the first workshop discussions were also convened for this second set of discussions. 
The groups were asked to consider a hypothetical benefits analysis. For the insect group, this was a specific 
problem on crucifers; the plant pathology group was given grapes; and the weed group was asked to conduct a 
benefits analysis on soybeans. The selection of crops was used to provide a breadth of crop types. Crucifers 
represent a limited-use crop with major resistance problems. Grapes are a perennial crop with a wide geographic 
distribution and a diversity of end products (juice, wine, table grapes, and raisins). Soybean, with new 
genetically engineered varieties, is a major field crop with wide distribution and both single and double-cropped 
plantings. 


Each group was asked to determine what information was critical, what data useful, what information 
was lacking, and what protocols could be established to ensure scientifically justified assessments. Groups were 
also asked to prioritize their suggested protocols and estimate how meaningful and achievable the resulting data 
should be for regulatory purposes. 


Plant Pathogens 


The group discussed two time lines, one where assessments are required in the short term, using present 
information, and the other where time to obtain experimental data is available. The group noted that we should 
be looking at impact assessments, and not assume benefits, per se. The group also felt that there were not 
significant differences when dealing with annual versus perennial crops.; i.e., the sources and kinds of 
information needed remain the same with respect to protocols. 


The group identified their assessment as involving fungicides on grapes. They identified a number of 
data sources including Fungicide and Nematicide Tests, refereed journals (i.e., Plant Disease, Phytopathology), 
experts (consultants, extension specialists, experiment station researchers), registrants, grower association, 
processors, and NASS. 


The following categories of information were deemed critical: profile of control alternatives (chemical 
and non-chemical), status of control alternatives (what is used, rates, costs), comparative efficacy information, 
and yield to quality information. There was a concensus that a system should be developed to determine the 
credibility or quality of information/data. At the same time, the group saw a definite lack of adequate 
information on yield and quality. 


Protocols can and should be developed which deal with the following focus areas: 


credibility of data 

field studies 

standard procedures for collection of use information, etc., from growers/advisors/consultants 
process for “quick” peer review 

statistical analyses of data 
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Benefits Protocols Workshop 10 


Insects 


The insect group used the situation of methomyl being cancelled for use on cabbage, with loopers as the 
target pest. The group identified a number of variables as essential for analysis (A), and important, but not 
critical to the analysis(B). 


The following tasks were seen as required for their benefits assessment: 


Identify alternatives (A). Determine the current market share of each pesticide or alternative being 
applied for this pest (A). Identify any imitations of alternatives (e.g., not registered in all states) (A). Assign 
current market share of methomyl to alternatives in proportion to their current market share (A). Identify 
unregistered alternatives that could be effective alternatives by next season (A) Conduct additional evaluation of 
unregistered alternatives (B) 


Evaluation of comparative performance (A). Use these data (if available) in order of priority: 

1.valid experiments taken to yield and/or quality endpoint, using data from last six years (try to find 20 
tests per growing region) 

2. valid experiments with efficacy data only combined with injury-loss model to estimate yield impacts. 
If time allows, conduct tests to validate observations from (1) and (2) (B) 

3. yield loss estimates based on expert opinion. Consult published state recommendations for states that 
use methomyl to control loopers on cabbage (B) 


Consult cabbage experts in states that use methomyl to control loopers on cabbage (Their input is 
especially valuable for: viability/limitations of alternatives, other alternatives (e.g., biocontrol), and other 
combinations of alternatives (e.g., tank mix, 2 alternatives applied during season) 


Consider resistance management (A). In this situation, one alternative (pyrethroids) has well-known 
resistance problems. Conduct an in-depth analysis of resistance (B) 


Consider the IPM value of target pesticide (A). Conduct an in-depth analysis of IPM considerations (B) 


Submit report to external peer review (A). Reviewers should include cabbage experts in affected states, 
affected registrants, food processors, and commodity groups. Revise report based on peer review comments (A) 


For every parameter it is important to assign some defined confidence value that indicates the reliability 
of the underlying data. The urgency of the situation will determine the amount of time allowed for analysis. In 
general, well-defined human risks are most urgent, followed by undefined human risks and ecological risks. 


The insect group recommends three tools to assist benefits analysis: 


1. Database of experts by commodity and pest. The analyst could readily identify contacts/possible data 


sources. 
2. Database of technical resources (where data can be found). 
3. Shell to evaluate comparative performance (fill in the blanks). 
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Weeds 


The weed group focussed on wheat rather than soybeans. Their benefits analysis was basically designed 
from the experience of the group, primarily that of Leonard Gianessi. 


Specific information that is considered critical or useful is indicated later. All information that is 
considered critical must be obtained for a successful assessment to be conducted. Information deemed useful 
should be collected to whatever degree possible. Assumed time for completing the assessment was 90 days. 


For the major crops, crop statistics and usage data are generally available and sufficient. For specialty 
crops, much more of the data will need to be gathered. Where data for a few important states is not available, 
efforts should be made to contact specialists in the state to obtain needed information. Where the data cannot be 
obtained within a reasonable amount of time, the information should be interpolated from nearby states. 


Where expert opinion is utilized, sources substantiating that opinion should be documented. What is not 
always available is expert opinion on infestations, recommendations, and efficacy of treatments (level of 
control). There is a need for databases of recommendations and infestation control levels. 


Review of model results is considered essential to account for factors outside of the models parameters 
(1.e., wild garlic quality loss, weed resistance and crop marketability). A review of results is needed by state 
experts, registrants, and commodity groups. 


The example used was an evaluation of phenoxys and sulfonylureas . A list of needed data and analyses 
is provided in Table 2 on the next page. 
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Table 2. List of critical information and processes for benefits assessments. 


Critical > 
Critical 
Useful > 
Useful 
Useful 
Useful 


Critical > 
Critical 
Useful 
Useful 
Useful 
Critical > 
Useful 
Useful 


Critical > 
Critical > 
Critical 

Critical 

Critical > 
Useful —> 
Critical > 


Crop Statistics from NASS 
area planted/harvested 
breakout by: 
State 
crop type - spring/winter/Durham 
irrigated/dryland 


Usage Data - interpolated to non-responding states 

NASS and Cal EPA 

state specialists 

growers, consultants 

NCFAP and EPA, industry 

Percent acres treated by A.I. 
Amount/formulation/timing/method 
Trends over 5-7 years 


Prices - from state reports 

Expert opinion on: 
acres infested by each pest above threshold 
level of control of pest by pesticide 

Run through NCFAP model 

Run economic study of marginal value of return 

Review of results 
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Discussion of Recommendations 


Based on the three groups and two discussion periods, there were four major 
focus areas selected as a point of departure for recommendations regarding protocols. 
Any protocols must address data (sources, types, priorities of use), quality assurance (for 
data and models/analysis), analysis/output (information provided, structure), and peer 
review (who, process, time, feedback). 


Several general issues were raised concerning the overall assessment process. Is 
there a need for protocols regarding the establishment of the assessment team? Should 
specific disciplines (i.e., economics or rural sociology) always be included? What are the 
impacts of time constraints (rapid turn-a-rounds)? Given the time constraints often 
imposed, an inventory of researchers and extension personnel organized or searchable by 
commodity and pest would be highly valuable. The ability to search this database by state 
and pesticide expertise would also be important. Some of this information is already 
available from specific organizations such as IR-4 and disciplinary societies. 


Data/Information Protocols. The initial discussion above led directly into 
information needs. Protocols should address the sources and types of information 
required for assessments. In addition to a specialist directory/database, information on 
pests, the damage they do, and control alternatives is obviously mandated. What are the 
sources of data and can we prioritize these data? 


Sources of data include experimental data, some published, but much of which 
can be found only in the ‘gray’ literature (reports to experiment stations, grower 
associations, companies) and in paper form in storage by individual scientists. Given our 
disciplinary approach, even published data are spread among a diverse journal base 
including regional, commodity and discipline-based minor publication sources. Decisions 
will need to be made regarding the use of electronic information (web pages, electronic 
journals, web-based databases). 


Regardless of source, protocols must address the issues of what information is 
available, identification of the source, and estimation of the quality. Priorities can be 
established for the weighting of information based on quality. For example, reviewed 
publications should carry more weight than survey results or scientific estimates. Even 
within comparisons of experimental data, experiments replicated across regions and years 
should be weighed more heavily than single experiments (see quality discussion below). 


Data Protocols should address the minimum types of information required. In 
general, pest data must include pest incidence/distribution, resistance problems, yield 
impacts from given levels of pests, pest interspecies competition (for weeds), and 
biotypes or strains. Crop data requires acreage, yields, variety information, quality 
measurements (where appropriate), soil characteristics, and market prices. Pesticide data 
must include efficacy, market share, alternatives, label information, formulations, 
application alternatives, and other pertinent use information. All of these data, in so far as 
possible, should be partitioned by state (or at least region) and commodity. 
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Benefits protocols Workshop 14 


Quality Protocols. Quality assurance has been a major issue in benefits assessments for 
regulatory decisions. For experimental data and databases, protocols should use some 
index which takes into account the amount and type of replication. The participants 
agreed that some form of what we termed the Borth Index (see summary of Discussion 
Session I) would provide an adequate quality measure for “hard” data. A similar 
approach could be used for questionnaires and surveys (by estimating the breadth and 
depth of questions and responders), and, perhaps, even scientific estimates (by assessing 
the experience and disciplinary diversity of the scientists). Criteria should be established 
for statistical analyses and models used in assessments based on the types of assumptions 
and the level of documentation. These protocols would be subject to public review and 
comment (and change with new methodologies and technologies). 


Analysis/Output Structure Protocols. With EPA as the primary client/customer, 
protocols, which specify the nature and form of analysis or model output, are crucial. By 
providing the types of information EPA needs (in a structured form which doesn’t require 
extrapolation or modification), considerable time and potential for error could be 
avoided. Standardization at this level is easier to achieve than for field experiments 
themselves. Such protocols also can be developed to help make the results easier for the 
non-scientist or non-specialist to understand. 


It is important that the end user groups define the format and information content. 
This may involve protocols which specify structures/content dependent on the end user. 
EPA and commodity organizations may need somewhat different analyses and/or results 
structure. Those developing the specific protocols should be sensitive to these differing 
needs. 


Review Protocols. These protocols should cover who is to be involved in the review 
process, what role each reviewer plays, how it is carried out, and how differences in 
reviews are to be handled. The workshop participants felt that all impacted parties should 
play a part in the review. EPA can play a critical role in the review process by providing 
feedback prior to the assessment submission. By fully involving EPA, the resulting 
assessment will be certain to answer specific questions EPA personnel may have 
regarding the given assessment and supporting data/documents. 


Workshop participants recommended that there be specific protocols to deal with 
responses to reviews and methodologies for conflict resolution. Methods need to be 
defined by which the assessment team is responsive to reviews. Suggestions and 
criticisms should not be ignored or automatically included. Where strong differences in 
interpretation exist, there need to be guidelines that provide for evaluation of these 
differing interpretations. 


Final Comments. Workshop participants felt that it is important to proceed with 
the establishment of specific protocols. USDA, working with EPA, should establish an 
ad hoc group for this purpose. There was general concurrence on the focus areas for 
protocols and the information/processes for which protocols are needed. Proceedings for 
this workshop were submitted to all participants for review and comment. 
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Benefits Protocols Workshop 15 


Speakers/Demonstration Topics 


A Historical Perspective on Crop Loss Estimation 
Charles Main, Plant Pathology 


This was a review of a program in North Carolina in the 1970’s and 1980’s which provided crop loss 
estimates for all crops in North Carolina. The program used surveys and scouting for the information source and 
was discontinued when funding was cut. 


Weed Management and HDDS ( Herbicide Decision Support System) 
Gail Wilkerson, Systems Science 


HDDS is a Windows-based decision support system. The output can provide per acre estimates of yield 
loss due to specific weed species and the economic benefits from the use of specific herbicides. 


| The NCFAP Assessment Model 
Leonard Gianessi, Economics and Policy 


The NCFAP approach uses available information on pests, crops and pesticides for benefits analyses. 
The processes of data selection, incorporation, and analysis were explained. Data sources were described. 


A Pesticide Benefits Assessment for Cotton 
Robert Nichols, Policy 


A recent assessment process for cotton pesticides was briefly described with emphasis on the types and 
sources of data used. Areas of missing information were also described. 
Weeds and Herbicide Benefits Assessments 


David Pike, Weed Science 


The variables necessary for benefits assessments for weed management were described, as well as a 
number of weed and herbicide-specific problems encountered in assessments. 


A Web-based Damage/Yield Loss Model/Data Inventory 
Ronald Stinner, Entomology 


A new web-based database effort was described. The database consists of abstracts from published 
literature detailing the impact of specific pest population levels or indices on yield. 
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Printed Information Provided to Participants 


Hutchins, Scott H.1995. Pest Control Technology Benefits Methodology Working 
Group — A Concept Paper. Unpublished 


Osteen, Craig, and Rob Esworthy. 1997. NAPIAP: Issues in estimating benefits of 
pesticides. Proc. Third National IPM Symp./Workshop. Feb 27 — March 1, 1996. 
Pp 134-135. 


Stinner, R. E. 1995. Development of model systems for benefits assessments of 
management tactics in integrated pest management. A Proposal to NAPIAP, 
ACPA, and EPA. From the Center for Integrated Pest Management. Unpublished. 


Stinner, R. E. 1997. Estimating biological benefits of pesticides for regulatory decision 
making. Proc. Third National IPM Symp./Workshop. Feb 27 — March 1, 1996. Pp 
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Pest Control Technology Benertits Methodology Working Group 


A Concept Paper 


Scorn! Hutchins 


DowElanco, 9330 Zionsville Road, Indianapolis, IN 46268 
(PH 11931-4975" FAX231 11337-4567) 


Introduction: 


The utility of Integrated Pest Management (IPM), including the value of pest 
control technology specifically, has not been adequately quantified -- at least to the 
extent that research and regulatory decisions require. Indeed, although the value of a 
specific technology or combination of technologies within a pest management scenario 
may be empirically assumed, a clear and consistent method for assessing benefits (vs. 


features) has not been developed and implemented. This has resulted in: 


e Aniinability to quantitatively and objectively assess risk AND benefits within a single 
equation, leading to the possibility of inappropriate regulatory action 

e Inadequate or unrealistic expectations regarding the acquisition and usefulness of 
product performance data to assess the “value” of technology 

e A fragmented approach to evaluation of IPM systems leading to poor research 
focus and unrealistic expectations as to overall versus situational value of IPM 

e Product and technology goals focused on perceived benefits rather than actual 


benefits 


A clearly defined methodology, including protocols for specific types of assessments 
must be developed, validated, and adopted in order to ensure USDA, EPA, and 
registrants of technology are working toward the same objectives. 
Purpose and Scope: 

To address the pervasive need for assessing the benefits of pest control 


technology, the establishment of a Pest Control! Technology Benefits Methodology 
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Working Group is proposed. The proposed membership/sponsorship within a core 
team would include USDA (specifically the National Agricultural Pesticide Impact 
Assessment Program [NAPIAP]), EPA (specifically the Biological and Economic 
Assessment Division [BEAD]), and the American Crop Protection Association (ACPA). 
In addition to these stakeholders and core planning groups, any number of third party 
research institutions or centers (e.g., Center for Integrated Pest Management) would 
play a critical role with developing the protocols required for comprehensive benefits 
method development. 


The scope of the proposed working group would include: 


e Identification and consensus of categorical benefits (e.g., Numan safety) that 
require valuation with developed methodology (a theoretical list of possible benefits 
that might require consideration is attached). 

e A technique or protocol for assessing value to a specified beneficiary. These 
methods should transcend obvious techniques and include novel biological and 
economic modeling approaches to assess both tangible and intangible benefits. 

e Integration of all the categorical benefits within an overall model to determine 
collective or net value. The output or product of the benefit assessment should 
“link” with the output or product of the risk assessment. 

e Techniques to incorporate probability and simulation modeling techniques as a 
means to incorporate uncertainty and develop sensitivity analysis will be critical for 


the decision-making and policy-setting utility of the risk:benefit models. 


Work Group Process: 
Once commissioned as a dedicated initiative for the sponsoring groups, a series 
of meetings to outline and manage the project would be necessary. The core team 


would consist of three key members from each organization with the charge to: 


e Systematically dissect the numerous issues associated with benefits assessment 
from all perspectives, especially those affectettby methodology shortfalls. 

e Partition the issues into categorical benefits with associated beneficiaries. Develop 
or commission a subteam of experts to review the specific issues associated with 


each benefit/beneficiary combination; subteams have accountability for 
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Page 3 


recommendations regarding protocol development and use. A final set of protocols 
and methodology must be agreed to by USDA, EPA, and ACPA for uniform and 
consistent implementation. 

e Establish of product or technology characterization guidelines that will integrate 
within the benefits methodology and provide utility to both registrants and regulatory 
agencies. 

e Develop expectations and requirements for continuous improvement of the 
methodology as new techniques evolve. Develop review and appeal guidelines to 
cover situations or circumstances where a product or technology does not “fit” within 


the established methodology. 


Summary: 

Pest control technology provides dramatic benefits to end users and society as 
a whole, but these benefits generally are poorly understood and rarely quantified. In 
order to adequately assess the net utility of any product or technology to all parties, 
however, some estimate of both risk and benefit is required. To address this ongoing 
need, a Pest Control Technology Benefits Methodology Working Group is proposed 
with core membership from USDA, EPA, and ACPA. The purpose is to develop a set 
of consistent and accurate protocols to assess benefits of technology as part of the 
regulatory and product advancement decision process, but also to demonstrate the 


value of environmental stewardship and IPM in ways heretofore not quantified. 
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NAPIAP: Issues in Estimating Benefits of Pesticides 


Craig Osteen 
National Agricultural Pesticide Impact Assessment Program, USDA 
Rob Esworthy 
U.S. Environmental Protection Agency 
Moderators 


This session focused on issues of estimating 
economic impacts of pesticide regulations. These 
issues are important to IPM because pesticides are 
important tools in many IPM programs. Pesticide 
regulations can reduce the options available for 
some IPM programs with undesirable pest control, 
environmental, and __ resistance-management 
consequences. These concepts can also be applied 
to analyzing the economic impacts of IPM 
adoption. 


EPA and USDA/NAPIAP have created a working 
group to review currently used economic methods 
of USDA and EPA pesticide benefit assessments 
because of questions raised about their quality. The 
ultimate purpose is to develop an improved set of 
guidelines for estimating the economic effects of 
pesticide regulatory actions. The pnmary questions 
of concem are: 

1. Are we trying to measure the night things? 

2. What methods to estimate economic effects are 
feasible, given restrictions on time, manpower, 
etc? 

3. Assuming that acceptable methods are being 
applied by USDA and EPA, are they being 
properly applied? 

4. Are there new methods that should be em- 
ployed? 


Economic Analysis in the 
Pesticide Regulatory Process 


Rob Esworthy discussed the role of economic 
analyses in risk-benefit comparisons under FIFRA 
Special Reviews and other registration decisions 
and in regulatory-impact analyses. In EPA, as well 
as NAPIAP, biologists and economists cooperate 
in the benefit-assessment process. The key 
elements in assessing the benefits of a pesticide 
used on a crop include: major pests controlled, 
chemical and/or nonchemical alternatives to the 
pesticide, and comparative performance of the 
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alternatives in terms of pest control and crop yield 
or quality. Ultimately, the economic analyses 
require estimates of the use of the pesticide in 
question and changes in yield, quality, and/or 
production cost associated with changing to 
alternative control measures. 


The Current Approach 


Conceptually, the assessment of benefits by USDA 
and EPA is the same as estimating the annual net 
efficiency loss of removing the pesticide from the 
market and switching to the best alternative control 
option. Monetary values generally are not 
estimated for health and environmental effects of 
proposed regulatory actions, which are considered 
in EPA risk assessments. However, the economic- 
impact estimates can be used to estimate cost- 
effectiveness of risk-reducing options. 


The standard framework for estimating the net 
economic effect is based on traditional Marshallian 
demand-and-supply curves. The supply curve 1s 
modified to reflect changes in yield and cost; price 
and quantity changes are estimated; and changes in 
consumer and producer surpluses are summed to 
estimate net effect. 


Partial budgeting (change in value of production 
plus cost change) is used to estimate net effect 
when price changes are expected to be negligible 
or data to estimate price changes are not available. 
A variation on partial budgeting 1s often used when 
yield or quality losses are difficult to value: pest 
control experts are asked to develop equally 
effective control options, and the net effect is 
estimated as the cost of the new option minus the 
cost of the current approach. 


Pesticide regulations can affect various groups 
differently. These so-called distributional effects 


are not obvious from the “net effect.” 
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Distributional effects estimated in assessments 
often include economic effects on purchasers of 
affected commodities, growers of affected 
commodities, users and nonusers of the regulated 
pesticide, regions where economic losses are 
particularly severe, and growers of other crops. 
Changes in commodity-program payments are also 
estimated, where appropriate, because they can 
shift the distribution of impacts. 


Several methods are used to address price effects 
and associated welfare effects: demand-and-supply 
elasticities in simple static-equilibrium models; 
mathematical (quadratic) programming models; 
and econometric simulation models, such as 
AGSIM, that account for simultaneous price, 
acreage, consumer, and producer effects for several 
crops. 


Comments by Panel 


Fred Kuchler argued that the economic effects of 
pesticide regulations would ultimately affect rents 
and values of land, a pnmary fixed factor of 
production. This link may be an important 
distributional effect because approximately 40 
percent of land in U.S. farms is rented. At one 
time, most farmers owned all the Jand they farmed, 
so separating this effect was not important. But a 
significant portion of farmland is now owned by 
people who do not farm. Share rents would be 
affected in the same years as effects of pesticide 
regulations on costs and yields occur. Potential 
renters would ultimately change their cash rent 
bids as changes in prices, yields, and costs became 
apparent. 


Jerry Carlson focused on some important costs 
typically neglected in the benefit assessment 
process: phytotoxic effects of replacement 
pesticides, changes in drift damage to adjacent 
fields, changes in resistance development for 
remaining pesticides, and changes in the variability 
or risk of crop yield. In addition, there can be 
effects on the value of human capital: regulations 
could force growers to use new, unfamiliar 
techniques and receive lower financial retums until 
they gain experience with them. Carlson felt that 
there were difficult tasks where improvement was 
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needed: (1) correctly estimating market shares of 
replacement controls and (2) estimating crop yield 
changes for different technologies in different 
regions by using experimental data. Two other 
important issues that need to be addressed are 
estimating changes in commodity-program 
payments and changes in unit prices of remaining 
pesticide products. 


Enk Lichtenberg argued for a different approach to 
estimating the effects of regulations and focused on 
issues of data and data quality. He argued that crop 
science data fit poorly into the traditional economic 
framework, and better results could be obtained by 
collecting data capable of supporting estimation of 
economic relationships directly. Such data could 
be collected through USDA Farm Costs and 
Returns Surveys or pesticide-use surveys. The data 
currently collected are not sufficient by themselves, 
however, and would need to be augmented to 
include such items as: (1) output (yield) 
information; (2) quantities of individual pesticides 
used; (3) quantities of other inputs used, such as 
fertilizers, labor, cultivation methods, other 
nonchemical control methods, etc.; and (4) prices 
of all of the above. Panel data that included both 
cross-sectional and time-series information would 
support the use of dual methods and estimation of 
supply and input demand curves. Cross-section 
data alone would support estimation of production 
functions directly. The damage-control approach of 
Lichtenberg and Zilberman could be used to 
estimate damage; such estimates would be useful 
to cross-check damage estimates of crop scientists. 


Erik Lichtenberg identified some other issues. 
First, assumptions of perfect competition (no 
individual buyer or seller can affect market price) 
may be invalid in some markets. Large buyers of 
agricultural commodities, such as grain marketers 
or food processors, could influence the prices that 
growers receive. In addition, national governments 
play an important role in marketing commodities in 
international markets. Second, it is not clear how 
effects on first-level purchasers of agricultural 
commodities transmit to effects on retail-level 
consumers, so that the “consumer effects” 
currently identified may relate to wholesalers but 
not retail-level consumers. 


wa ev} 
bhicg torre Sere sarin tS) ‘bee ab a 
yet ai: dleortieay iad nun She i et 
re ry Aieb ne wt ’ get a pltelieet 
oi jee She git ok Page ede Sn siodgey ttt 


sive (tees i mtyehete GMS 
Ti iAL f } } fitj P ih a | ¢ ri Dewy wi? sree My 
med a! to ieee 


ot hereon aha ee 
ifthe woot Pan eee tgp Oks aed wil Bar ees 
pia ituat Guys GOL oe NT Ne Wings Want “te ere 
dant!) 20) Maret eS ea sD or ee 
ates eh Higoe aie p iibinl wie oh +) DEDIEA 
la pow Mites cacetia Yo alhega ! ale? gam he’ 


thor ahids | Le eh viv ey yy a) aw 19 


oo i at OGY “Spore! e ify 20 
gh adh 2gas tetas - thr) WO Keay ty eae a 
ie Deena yet tevin! aie bind 3 Wish 

(obey aadinn ke) OR hfgove fuii\, oy oman 

Uy te } ey a) at cts) ii surat 

CM .iebe fEh tira We Se + Pua cue 


y. apace be) Beat 


sé dons. bevei) Jara) 
+ 


ain mares OP 1 nail (eset pil 
anil ET bar, a a Det (OLB Utrete v 
ic Bers dliile anh Mapt esi. alt eto 
hc. ohana le ehh! eT bs Faas Sayin ioe He OY 
fey maki hk tf) epee Ge abhi mantis vl Teegeme 
ipicgae eae 2d WOOO THis tan cages 
sped eRe; TO ibys ey ER ee hinaw teylaebpn 
(Gite tip reg SL ey panini’ 
nm fea at Gliese «amie Boe fr <ycaliha 
hen st bien melita ageamh s ees 
tr edl-ve Choe ib SanattrrnneRyea TaN A ‘ wa et 


ayuces cola nieuw bonny & waned ed tbl 
Ow} doves pa tn Selenide YO 4 
fasha daar ykis qsonelion «6g abirae 
16 vagyud begin) ee RD! r ligand ad Weve 
praisfucn cite eae J RRO? latinos! 
rer aaung 2 oma tides saakangow), boul 
cliente teay, teeroae ceili oh FeO erro 
1s gehity citar grinder anata ara ih yey 
word wai ton end | vereed alhalais iene 
imaiwsign Wo emstedtomg, (pies § 
favabliaeve ao Modi a) Groene 
“spestts voranen) oi apvit am 
md riofansicabe OF Males qn alti abi 


‘en Spraarmang RAN LIO Pe Ales : 
Np ao uly armees 





































hi thee al heehee 


: penn ty iceing “eo oniet Une 
er a Lots hii 


dv tent unde ete rat sae 
es iycih YW «cw ror ra y 


mh jini ep (aay 0b pee |! i we ie ott? 7 
» ue. GAG! SMart 2 ip? ge pp! 
gucci dey fies LE etd) al Nets 7 
, 
Esl Ce ae a, ie od pi» ufleGhAg'’ ase: 
mi _ wy hiiasteo Ae enna = Pes ie 
Ae. ee pot? ft altri. 
eh ayy ay th mrer Couitilcoiy) i’ Lye te ie 
an id. :” ppedyhet Reta. sr TetedaGe | 
eine Aue Visitas" “Ree: oe: ri : 
Pasi 8 vt Vergo lriny rer (ah cation). , EDF 
r - 
> 
Lanttyl edad? n 
5 pba sainiatis, 8 Ter Tepeets dagot hart 7 
ei a eilin eae: Moe V he Oleh al diy : 
ee et a a the MO” ceo i = 


igwonh of i oer Falieeh neh 
‘ar heat yr i waa? Vimin Lereeethaal oats 
ee aN aisha ben [an nm hohe grein 
Awadvenst tool? Seah rte Ite ane anal tvs 
Ce hort » fONT Sat sehr : iM ; 
en tense ape 4 ba snl te et a Ay ext qle 
‘at Oe TA “ orien At arte? ofeey 
sy poaenve We agp’ ee: Altidruotie : 
ite, et ois etn ale ; ena Chih 

hog Mae 2 OG 
eases (ile otha 
























SSH e COC el PR PAS OE GANS RL 


Kstimating Biological Benefits of Pesticides for Regulatory Decision Making 


Ron Stinner 
North Carolina State University 
Moderator 


Introduction 


The National Agricultural Pesticide Impact 
Assessment Program (NAPJAP), a USDA/State 
program, was established in 1976 to promote 
informed regulatory decisions on agricultural 
pesticides. NAPIAP develops and distributes 
science-based information evaluating the benefits 
of pesticides in U.S. agricultural production. The 
information in NAPJAP assessment documents 1s 
provided to the U.S. Environmental Protection 
Agency (EPA) for use in its regulatory decision- 
making process. These documents also provide 
useful information to the USDA, agricultural 
scientists, and commodity groups. In February 
1995, a panel reviewing NAPIAP cniticized the 
program for using excessive “expert opinion” 
(scientific estimates) in lieu of documented 
biological data in these assessments. At the same 
time, the benefit-assessment process has suffered 
from a lack of protocols that could be used to guide 
the acquisition of such data. In an effort to better 
refine the benefit-assessment process, a Benefits 
Assessment Protocols Working Group was formed 
in 1995 to address these issues. The Working 
Group consists of representatives from USDA, 
EPA, and the American Crop _ Protection 
Association (ACPA). This workshop is the first 
result of the ongoing discussions on_ the 
development of assessment protocols. 


The panel participants have all had experience with 
NAPIAP and the benefit-assessment process. Drs. 
Jenkins and Pike are the NAPIAP State liaison 
representatives for their respective States and have 
also participated in the assessment process. Dr. 
Bridges was a member of the panel that reviewed 
NAPIAP; he also has done an assessment, using an 
innovative approach, of the benefits of pesticide 
use In peanut production. 


Panel Presentation 


Dr. Jenkins discussed the Pesticide Benefits 
Assessment Model, developed at Ohio State 
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University. This model attempts to assess the true 
economic impact of alternative control strategies 
and to provide information useful to regulatory 
decision making. The advantages of such an 
approach are: improved credibility and reliability, 
less expert opinion, consistent framework, and the 
development and use of formalized models. He 
also discussed the data needs and sources presently 
available. 


Dr. Bridges pointed to the major problems with the 
present benefit-assessment process: imbalance in 
risk and benefits (with large sums spent on risks 
and little on benefits), credibility (isk well-defined 
with systematic approach to assessment; benefits 
more diffuse and difficult to define), little 
investment in benefits methodology, and an 
underestimation of the importance of biological 
components and their variability. This 1s true for 
agribusiness as well as government regulators and 
university cooperators. Dr. Bridges recommended 
that NAPIAP develop a common ground for 
assessments that includes: (1) multiuser databases 
of pest occurrence (and damage) and demographics 
of pest-management practices and (2) common, 
consensual, and systematic processes for 
assessments. 


Dr. Pike addressed the history of assessments, 
noting that there has always been a balance of both 
expert opinion and empirical evidence, with the 
pendulum now moving away from expert opinion. 
He noted that in spite of regional variations and 
requirements, NAPIAP should be able to develop 
a set of protocols that include subjectivity; that is, 
both models and individuals to interpret the 
information (model, expert opinion, and empirical 
data). 


Discussion and Conclusions 


Numerous questions were raised, such as: How do 
you estimate the costs of practices (e.g., resistance 
management, new-product costs, and value of 
product alternatives)? This question led to a 
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discussion of individual costs versus averaging and 
the value of pnor knowledge (e.g., we know that 
curative methods always produce a higher return 
than prophylactic treatments when we average, but 
not necessarily when looking at individual years 
and fields). 


Where are the data? Can we realistically estimate 
yield as related to damage indices? Are such 
models well known, and more important, are they 
transparent (1s it obvious what they do)? This 
discussion led to a major conclusion that the 
concept of transparency was cnitical to the benefit- 
assessment process. A main concer with expert 
opinion 1s how interpretations are made from point 
A (data or estimates) “to. point. .B 
(recommendations). If the entire logical process 
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from A to B is made clear (hence the term, 
transparent), then it stops being expert opinion and 
becomes empirical information. Because yield- 
quality effects are the most difficult to estimate, 
models become necessary tools. However, the 
inherent complexity and variability of our 
agricultural system demand that any model results 
be interpreted and analyzed in light ‘of this 
variability. 


The workshop concluded with the consensus that 
NAPIAP should develop protocol criteria that 
include the use of transparent models and careful 
analysis while not forgoing expert opinion. All 
affected parties should be a part of the 
development of these protocols. Benefit 
assessment should be an integral part of product 
development. 
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Development of Model Systems for 
Benefits Assessments of Management Tactics 
in Integrated Pest Management 
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INTRODUCTION 


Current regulations require that benefits and risks of 
products be assessed when pesticides are evaluated for continued 
Pegistration of fereq stration. Risk assessment protocols are 
well sdocumenlecwewren botharegulatorsnand andustry understanding 
Woat must be accomplished to provide a risk assessment. The 
Denielits’ Betdescintne! squetion, “however, is much more complex 
Sno eene enderest lt on a) benetits assessment” much more 


probabilistic, SHdsstibyece tonmer#tiersme, ,Thisipzoeposal seeks: to 
define an approach to the development of benefits assessment 
methods¥protocolse: Tercoestmoteaddaréss théiissuce of defaning the 


Speettic provocols, bue rather the process to be uséd and some of 
the information required. 


EPA, NAPIAP, and ACPA all have a vital stake in benefits 
Sesessmente Ther NatwonaiwAguicultural —pesticide) Impact 
Assessment, Program: (NAPIAP); a UeS: Department of Agriculture 
(USDA) /State program, was established in 1976 to promote informed 
regulatory decisions on agricultural pesticides. NAPIAP develops 
and distributes science-based information evaluating the benefits 
SGAPes 1 OfdGsS 2nMUGS) Sorreukturak productions  Thelantormation 
in NAPIAP assessment documents is provided to the U.S. 
Environmental ProtectaonsAgency® (BPA) sfor use im their 
regulatory, decision-making process. Because these decisions are 
Clhicicaim@eoumemoecesOTmAGCPAy thei perznduUtsinto? thaseprocess! should 
be encouraged. 


Im vPebruary 1995, a panel reviewing NAPIAP criticized the 


Program Lor Using excessive “expert opinion” (scientific 
estimates) in lieu of documented biological data in these 
assessments. The benefits assessment process has suffered froma 
tack) @GeprobecolSeawhichscoutdjbesused toyquide theaacquasitaoneot 
such data. In an effort to better refine the benefits assessment 
process, a Benefits Methodology Working Group was formed in 1995 
to address these issues. The Working Group consists of 


representatives from USDA, EPA, and the American Crop Protection 
ASsociarxzon s (ACPA): 


Currently the assessment of benefits focuses on producers 
and, to a limited extent, on consumers from an economic 
perspective. There are other significant aspects of benefits 
that merit consideration, such as the availability of an abundant 
and varied food supply free from pests and the hazards imposed by 
chemicals that pest infestations induce (e.g., phytoalexins and 
mycotoxins). These health related benefits are complex, and a 
great deal of effort would be required to refine approaches to 
characterize and measure them. 
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APPROACH 


Puts les omCOUrIMuagtt on Ol a project by the same title. The 
Process of protocols development has five phases: 


Te Be INCPOstirOonmEOomCest rec protocol! goals 
IT. Assessment of data needs 
Jie Wveve LOuMen Ola, MOde.e inventory (database) 
ive CoOllecoionvand integration of additional databases 
Vo AGtUel = protocol Gqevelopmentc. 

Based on the above information, protocols could be set in 
place which clearly define the product-specific information 
needed to provide acceptable benefits assessments. Phases I and 
II have already been addressed by the Benefits Methodology 
Working Group, as outlined below under the initial year of 
Pati home toi so coposal.s ~The focus of this proposal iS the 
collection of a Model Inventory and online searchable database. 

Elucidation of Protocol Goals 

A series of meetings on the Benefits Methodology Working 
Group has already defined the desired goals for the group 
members, from their perspectives, as outlined below. 

ACPA: 

° Need to establish clear expectations from regulatory 
agencies on what is required and why with regard to product 
benefits 


° Understand how product attributes relate to product benefits 


° Advance product candidates based upon an accepted 
Push: benefit 1atio 


° Meet/exceed expectations of regulatory agencies for 
submission of benefits data 


° Tmorove, Ghewlmage of agricultural products and obj ectively 
describe the value of new technology and IPM 


° Describe tangible and intangible benefits of products. 
USDA: 
° Establish transparent objectivity in the assessment of 


product or technology benefits, much like risk assessments 
do today 
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° Develop protocols and techniques to quantitatively assess 
product benefits without dependence on specific comparative 
broduem Studresiin tne i veld 


EPA: 

° Develoo -cCredibisguey behind benefits assessments in order to 
allow tobvecurve traceott decisions with risk @esessments 

. Weve OprrCatiet le expec laciols Lor collectron and “use of 
DECGUCe permrormance Gate andwlink this’ to damage or Grop 
loss models 

° Organize and utilize existing data on product performance 


fOr Cilrrene wand anticipated regulatory actions. 
ASSESSMENT OF DATA NEEDS 


The Benefits Methodology Working Group has held several 
meetings to “discuss data needs. In addition, NAPIAP funded 
fenrough the Center “for 1PM) "a workshop (Bstimating Biological 
Benefits of Pesticides for Regulatory’ Decision Making) ‘at the 
National IPM Symposium/Workshop in February of this year. The 
workshop consisted of three panel speakers and a lengthy 


ObecicstOn wmlnempane! ancluded/Drs. David Pike, Jeff Jenkins, 
and David Bridges. The panel participants all had experience 
with NAPIAP and the benefits assessment process. Drs. Jenkins 


and Pike are the NAPIAP State Liaison Representatives for their 
respective states, and have also participated in the assessment 
process. Dr. Bridges was a member of the panel that reviewed 
NAPIAP; he has also done an assessment, using an innovative 
approach, of the benefits of pesticide use in peanut production. 


A proceedings will be published shortly for the Symposium/ 
WoEekSnope rhe "discuse von lech tovtne conciisiron that "the concept 
of transparency was critical to the benefits assessment process. 
A main concern with expert opinion is how interpretations are 
made from point A (data, estimates) to point B (recommendations). 
Te oie ontire tLogical process strom a to ‘Bs ‘made clear (hence 
the, term, cransparent) then it stops being expert opinion and 
becomes empirical information. Because yield/quality effects are 
the most difficult to estimate, models become necessary tools. 
However, the inherent complexity and variability of our 
agricultural system demands that any model results be interpreted 
aria tanalbyzed im Light "or “this varrabi lity: 


The workshop ‘concluded with the ‘consensus that NAPIAP should 
eevelop vrococol! Criteria which include the use of ‘transparent 
models and caretul analysis, while not forgoing expert “opinion. 
All affected parties should be a part of the development of these 
protocols. Benefits assessment should be an integral part of 
product development. 
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There was Significant disagreement in both the Workshop and 
Ene wWOrking 8GrouD ass Ostia availability of the models required 
for such transparent analyses. 


Development of a Model Inventory 


TiesCO) LeCtLl OuponOmOrGalizatLon Of a Model einventory 1s a 
complex task requiring consideration of all the data needs. For 
even simple benefits assessments, two major categories of models 
(Or data) are required: (1) density-damage models and .(2) pest 
occurrence information. 


Density-Damage Models (Single Species). This group of models 
ere generally Simple, usually crop= and site-specific regression 
models. .Most of these models are “single. species” descriptions 
for insect and disease damage. Work on cataloging these models 
feasvalready ibequn., Dm. Rosanne Minarovic, .a crop: scientist; is 
Peawchi no euewNColaliiorary, which has an extensive agraculturas 
FOUrMalSs Col fection. Papers containing these models are being 
copied and Model Inventory Data Forms (MIDFs) prepared for each 
model These MIDPa (contain sintormation on .the crop, pest, 
locaton, pesticides.used (1f pertinent), and necessary model 
Gece OL Lome. 


The MIDPSe sare «inva Stendardized format and will be -stoned 
for the present as WordPerfect files (see Model Inventory 
Database Development). 


Density-Damage Models (Multiple Species). Because weeds 
generally occur in multiple-species associations, and because 
weed yield loss is not necessarily additive, weeds damage 
requires a multi-species approach not handled by standard 
statistical models. There are a number of other models, such as 
HERB (Coble et al.), which are already available for examining 
economic benefits of varying weed management tactics. These 
models are beyond the scope of this proposal. I have, however, 
contacted Dr. Coble and we will cooperate in any way necessary to 
integrate these models with our “single-species” inventory. 


Pest Occurrence. Having an inventory of density-damage 
models is only the first half of the equation. We must also have 
detailed information on pest occurrence and demographics. That 
is, tomwassess total damage’ (and thus economic benefit fErom 
control), we must use both the damage resulting from given 
Ceneitles and mow orten these, densities occur in a specific 
region. This information (where, frequency, levels of 
infestation) will have to be gleaned from NAPIAP surveys, 
literature reviews, and expert opinion (documented where 
possible). 


Model Inventory Database Development 


DRiIMormetherinformation anventorieda will be stored initially 
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as WordPerfect and text documents. In order to efficiently and 
effectively search (and integrate) the model inventory, the 
WordPerfect “Model Inventory Data Forms” will be translated into 
an SOlU-Ccompliantl, relational database, such as Microsoft Access. 
We are currently installing a new Windows NT server and could 
Sacily IMake the database accessible to authorized individuals via 
the Internet using Web-based forms. The database could also be 
macopavelleplestnrougieeD-ROM Or direct FTP. If funds are 
avetleble for thes. Pech Occtirrence” inventory, that effort would 
include programming for on-line compilation of state/regional 
benefits, based on average pest occurrence and resulting damage. 
The databases will also be compatible with the new “Pest 
Management Information/Decision Support System” being developed 
by CSREES and NAPTAP with Terry Janssen at Argonne National Lab. 


Collection and Integration of Additional Databases 


Phaser i Viwila only be initially investigated in this 
Project. elit ocewil labe. .arconeinuation of a project under the same 
eta er 


There are numerous complex decision aids and crop simulation 
mouels which could wholly or partially veplace site-specific 
mocelc wand fi liestate or reqiomal data gaps. Identification, and 
review of these models will be a time-consuming process, but 
peuldibe ercruticalss boosts to methodology protocols. “Resulting 
databases would need to be integrated into the model inventory 
database. 
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